Polynomial Time Algorithms for Branching Markov Decision Processes and Probabilistic Min(Max) Polynomial Bellman Equations

We show that one can approximate the least fixed point solution for a multivariate system of monotone probabilistic max(min) polynomial equations, referred to as maxPPSs (and minPPSs, respectively), in time polynomial in both the encoding size of the system of equations and in log(1/epsilon), where epsilon > 0 is the desired additive error bound of the solution. (The model of computation is the standard Turing machine model.) We establish this result using a generalization of Newton's method which applies to maxPPSs and minPPSs, even though the underlying functions are only piecewise-differentiable. This generalizes our recent work which provided a P-time algorithm for purely probabilistic PPSs. These equations form the Bellman optimality equations for several important classes of infinite-state Markov Decision Processes (MDPs). Thus, as a corollary, we obtain the first polynomial time algorithms for computing to within arbitrary desired precision the optimal value vector for several classes of infinite-state MDPs which arise as extensions of classic, and heavily studied, purely stochastic processes. These include both the problem of maximizing and mininizing the termination (extinction) probability of multi-type branching MDPs, stochastic context-free MDPs, and 1-exit Recursive MDPs. Furthermore, we also show that we can compute in P-time an epsilon-optimal policy for both maximizing and minimizing branching, context-free, and 1-exit-Recursive MDPs, for any given desired epsilon > 0. This is despite the fact that actually computing optimal strategies is Sqrt-Sum-hard and PosSLP-hard in this setting. We also derive, as an easy consequence of these results, an FNP upper bound on the complexity of computing the value (within arbitrary desired precision) of branching simple stochastic games (BSSGs)

Financial instability from local market measures

We study the emergence of instabilities in a stylized model of a financial market, when different market actors calculate prices according to different (local) market measures. We derive typical properties for ensembles of large random markets using techniques borrowed from statistical mechanics of disordered systems. We show that, depending on the number of financial instruments available and on the heterogeneity of local measures, the market moves from an arbitrage-free phase to an unstable one, where the complexity of the market - as measured by the diversity of financial instruments - increases, and arbitrage opportunities arise. A sharp transition separates the two phases. Focusing on two different classes of local measures inspired by real markets strategies, we are able to analytically compute the critical lines, corroborating our findings with numerical simulations.Comment: 17 pages, 4 figure

Optimal Investment in the Development of Oil and Gas Field

Let an oil and gas field consists of clusters in each of which an investor can launch at most one project. During the implementation of a particular project, all characteristics are known, including annual production volumes, necessary investment volumes, and profit. The total amount of investments that the investor spends on developing the field during the entire planning period we know. It is required to determine which projects to implement in each cluster so that, within the total amount of investments, the profit for the entire planning period is maximum. The problem under consideration is NP-hard. However, it is solved by dynamic programming with pseudopolynomial time complexity. Nevertheless, in practice, there are additional constraints that do not allow solving the problem with acceptable accuracy at a reasonable time. Such restrictions, in particular, are annual production volumes. In this paper, we considered only the upper constraints that are dictated by the pipeline capacity. For the investment optimization problem with such additional restrictions, we obtain qualitative results, propose an approximate algorithm, and investigate its properties. Based on the results of a numerical experiment, we conclude that the developed algorithm builds a solution close (in terms of the objective function) to the optimal one

Heat release by controlled continuous-time Markov jump processes

We derive the equations governing the protocols minimizing the heat released by a continuous-time Markov jump process on a one-dimensional countable state space during a transition between assigned initial and final probability distributions in a finite time horizon. In particular, we identify the hypotheses on the transition rates under which the optimal control strategy and the probability distribution of the Markov jump problem obey a system of differential equations of Hamilton-Bellman-Jacobi-type. As the state-space mesh tends to zero, these equations converge to those satisfied by the diffusion process minimizing the heat released in the Langevin formulation of the same problem. We also show that in full analogy with the continuum case, heat minimization is equivalent to entropy production minimization. Thus, our results may be interpreted as a refined version of the second law of thermodynamics.Comment: final version, section 2.1 revised, 26 pages, 3 figure

Optimal Investment-Consumption Problem with Constraint

In this paper, we consider an optimal investment-consumption problem subject to a closed convex constraint. In the problem, a constraint is imposed on both the investment and the consumption strategy, rather than just on the investment. The existence of solution is established by using the Martingale technique and convex duality. In addition to investment, our technique embeds also the consumption into a family of fictitious markets. However, with the addition of consumption, it leads to nonreflexive dual spaces. This difficulty is overcome by employing the so-called technique of \relaxation-projection" to establish the existence of solution to the problem. Furthermore, if the solution to the dual problem is obtained, then the solution to the primal problem can be found by using the characterization of the solution. An illustrative example is given with a dynamic risk constraint to demonstrate the method

Option prices under Bayesian learning: implied volatility dynamics and predictive densities

This paper shows that many of the empirical biases of the Black and Scholes option pricing model can be explained by Bayesian learning effects. In the context of an equilibrium model where dividend news evolve on a binomial lattice with unknown but recursively updated probabilities we derive closed-form pricing formulas for European options. Learning is found to generate asymmetric skews in the implied volatility surface and systematic patterns in the term structure of option prices. Data on S&P 500 index option prices is used to back out the parameters of the underlying learning process and to predict the evolution in the cross-section of option prices. The proposed model leads to lower out-of-sample forecast errors and smaller hedging errors than a variety of alternative option pricing models, including Black-Scholes and a GARCH model